Dyadic Decision Trees

These notes introduce a new kind of classifier called a dyadic decision tree (DDT). We also introduce a discrimination rule for learning a DDT that achieves the optimal rate of convergence, ER(ĥn) − R∗ = O(n−1/d), for the box-counting class, which was defined in the previous set of notes. This improves on the rate of ER(ĥn)−R = O(n−1/(d+2)) for the histogram sieve estimator from the previous notes. Dyadic decision trees are based on recursively splitting the input space at the midpoint along some dimension. This is in contrast to conventional decision trees that allow the splits to occur at any point. Yet DDTs can still approximate complex decision boundaries, and the restriction to dyadic splits makes it possible to globally optimize a complexity penalized empirical risk criterion, in contrast to mainstream methods for decision tree learning that first perform greedy growing followed by pruning. These notes will not discuss implementation of the discrimination rules, but the interested reader can find algorithms and computational considerations discussed in [1, 2, 3].